SOME COUNTEREXAMPLES IN DYNAMICS OF RATIONAL SEMIGROUPS 
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Abstract. We give an example of two rational functions with non-equal Julia sets that generate 
a rational semigroup whose completely invariant Julia set is a closed line segment. We also give 
an example of polynomials with unequal Julia sets that generate a non nearly Abelian polynomial 
semigroup with the property that the Julia set of one generator is equal to the Julia set of the 
semigroup. These examples show that certain conjectures in the field of dynamics of rational 
semigroups do not hold as stated and therefore require the allowance of certain exceptional cases. 

1. Introduction 

In [SJ, Hinkkanen and Martin develop a theory of dynamics of rational semigroups as a gener- 
alization of the classical theory of the dynamics of the iteration of a rational function defined on 
the Riemann sphere C. In that paper and in subsequent communications, they put forth several 
conjectures, some of which will be addressed here. In particular, we provide counterexamples to 
Conjectures 11.11 11.21 and 11.41 In light of these examples the conjectures are then suitably modified 
and as such remain open questions. We begin by developing the necessary background to state these 
questions. 

In what follows all notions of convergence will be with respect to the spherical metric on C. A 
rational semigroup G is a semigroup of rational functions of degree at least two defined on C with 
the semigroup operation being functional composition. (One may wish to allow some or all of the 
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maps in G to be Mobius, for example, when one is considering Kleinian groups as in [8], but since 
the examples constructed here all contain maps of degree two or more, we will use our simplified 
definition to avoid any technical complications which are not pertinent to this paper.) When a 
semigroup G is generated by the functions {/i, /b, ...,/„,... }, we write this as 

G = (fl, /2, . • • ,/n, • • •)• 

On p. 360 of [3J, the definitions of the set of normality, often called the Fatou set, and the Julia 
set of a rational semigroup are as follows: 

Definition 1.1. For a rational semigroup G we define the set of normality of G, N(G), by 

N(G) = {z 6 C : 3 a neighborhood of z on which G is a normal family} 
and define the Julia set of G, J(G), by 

J(G)=C\N(G). 

Clearly from these definitions we see that N(G) is an open set and therefore its complement J(G) 
is a compact set. These definitions generalize the case of iteration of a single rational function and 
we write N((h}) = Nh and J{(h)) = Jh- Note that J{G) contains the Julia set of each element of G. 
For research on (semi-)hyperbolicity and Hausdorff dimension of Julia sets of rational semigroups, 
see QUI HU US]. 

Definition 1.2. If h is a map of a set Y into itself, a subset X of Y is: 

i) forward invariant under h if h(X) C X: 
ii)backward invariant under h if h~ 1 (X) C X; 
Hi) completely invariant under h if h(X) C X and h^ 1 (X) C X. 



SOME COUNTEREXAMPLES IN DYNAMICS OF RATIONAL SEMIGROUPS 3 

It is well known that for a rational function h, the set of normality of h and the Julia set of h are 
completely invariant under h (see [2], p. 54), i.e., 

(1.1) h(N h ) = N h = h~\N h ) and h(J h ) = J h = h-\j h ). 

In fact, the following property holds. 

Property 1.1. For a rational map h of degree at least two the set Jh is the smallest closed completely 
invariant (under h) set which contains three or more points (see [J], P- 67). 

From Definition ll.il it follows that N(G) is forward invariant under each element of G and, thus, 
J(G) is backward invariant under each element of G (see [3], p. 360). The sets N{G) and J(G) are, 
however, not necessarily completely invariant under the elements of G. This is in contrast to the 
case of single function dynamics as noted in (jl.ip . However, one could generalize the classical notion 
of the Julia set of a single function in such a way as to force the Julia set of a rational semigroup to 
be completely invariant under each element of the semigroup. Thus, we give the following definition. 

Definition 1.3. For a rational semigroup G wc define the completely invariant Julia set of G 

E(G) = f^iS : S is closed, completely invariant under each g E G, if(S) > 3} 
where #(5) denotes the cardinality of S. 

We note that E(G) exists, is closed, is completely invariant under each element of G and contains 
the Julia set of each element of G by Propertv ll.il 

Definition 1.4. For a rational semigroup G we define the completely invariant set of normality 
of G, W(G), to be the complement of E(G), i.e., 

W(G) =C\E(G). 
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Note that W(G) is open and it is also completely invariant under each element of G. 

We state the following conjectures which are due to A. Hinkkanen and G. Martin (see [7]). 

Conjecture 1.1. If G is a rational semigroup which contains two maps f and g such that Jf =/= J g 
and E(G) 7^ C, then W(G) has exactly two components, each of which is simply connected, and 
E{G) is equal to the boundary of each of these components. 

Conjecture 1.2. If G is a rational semigroup which contains two maps f and g such that Jf 7^ J g 
and E(G) 7^ C, then E{G) is a simple closed curve in C. 

In section [5] we give a method for constructing functions (as well as providing concrete functions) 
whose Julia sets are unequal, but which generate a semigroup whose completely invariant Julia set 
is a line segment. Hence the above conjectures do not hold. But since the only completely invariant 
Julia sets of rational semigroups which are known at this time (when the semigroup contains two 
maps with unequal Julia sets) are C (see [5] and [7J) or sets which are Mobius equivalent to a line 
segment or circle, the authors put forth the following conjecture, which is currently unresolved. 

Conjecture 1.3. If G is a rational semigroup which contains two maps f and g such that Jf 7^ J g 
and E(G) is not the whole Riemann sphere, then E(G) is Mobius equivalent to a line segment or a 
circle. 

Remark 1.1. We briefly explain some evidence that compels us to pose Conjecture 11.31 in this way. 
Our example of a rational semigroup G with E(G) being a line segment is rigid since G contains 
a Tchebycheff polynomial, which is known to be postcritically finite (and hence, rigid). On the 
other hand, an example of a rational semigroup G with E(G) being a (unit) circle generated by 
rational functions /1, • ■ • , /« with non-equal Julia sets is easily constructed by choosing finite Blaschke 
products as the /j's. However, it seems difficult to quasiconformally deform /1, • ■ • , /« simultaneously 
so that the completely invariant Julia set of the resulting rational semigroup is not a circle. 
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In section [3] we provide a counterexample to the following conjecture also due to Hinkkanen and 
Martin [J. 

Conjecture 1.4. Let G be a polynomial semigroup such that Jh = J{G) for some h S G. Then 
Jf = J g for all f,g € G (and hence G is nearly abelian by Theorem \3.2\) . 

In our counterexample J(G) is a closed line segment. Since no other types of counterexamples 
are known, we modify this conjecture as follows and note that it remains unresolved. 

Conjecture 1.5. Let G be a polynomial semigroup such that J] x — J{G) for some h 6 G where 
J{G) is not a line segment. Then Jf = J g for all f,g € G ( and hence G is nearly abelian by 
Corollary [EM- 

2. Counterexamples to Conjectures ll.ll and 11.21 

We begin this section with some notation and lemmas. Let 4>{z) — z z ^\ and denote the upper 
half plane as U — {z : 3z > 0}. Then </> maps U one-to-one onto Q = C \ [—1, 1] and <fi maps R 
two-to-one onto / = [—1, 1]. We call a map / odd if f(—z) = —f(z) and we call a map / even if 

/(-*) = /(*)• 

Lemma 2.1. A function f is an odd rational map such that f(U) = U if and only if it has the form 



h N n 

(2.1) /(,,-„- -Ej^- 

where a, 6, Aj, Bj > 0. 

Proof. Let / be an odd rational map such that f(U) — U . Then any preimage of infinity must be real 
(else there would exist a preimage of infinity in U) and simple (else there would be points in U that 
map outside of U). Again, since f(U) — U, it follows that / must be of the form az— j — Ylj=i z -a 
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where a, b, Cj > and ay € R. Since /(R) = R and f{—z) — —f(z) we conclude that the poles other 
than the one which might possibly exist at the origin must come in pairs of real numbers symmetric 

about the origin. Hence f(z) — az — - — V W , — — V W , — r 2 — where 6,- > 0, which can be 

algebraically reduced to (|2.1| • 

Let / be a map of the form (|2.ip . Hence / is odd, rational, maps U into U (since each term in 
the sum does), and maps R into R (since the coefficients are all real). From this it easily follows 
that f(U) =U. □ 

Lemma 2.2. Let f be a rational map. Then [f{z)} 2 is even if and only if f is even or odd. 

Proof. Suppose [/(z)] 2 = [/(— z)] 2 . Then an analytic square root of [/(— z)] 2 (defined locally away 
from the zeroes and poles of /) is either f(z) or —f(z). The identity theorem can then be used to 
show that f(—z) is either f(z) or —f(z) globally, i.e., / is either even or odd. 

The reverse implication is immediate. □ 

Lemma 2.3. Let f be a rational map. Then <f> o / is even if and only if [f{z)] 2 is even. 

Proof. Since 4>{z) — tp{z 2 ) for ip(z) — we see that [/(z)] 2 = o cj)(f(z)) and tfj' 1 o 4>{f{— z)) = 
[/(— z)] 2 . The lemma easily follows. □ 

Lemma 2.4. If g is an even rational function, then g{z) = h(z 2 ) for some rational map h. 

Proof. For z ^ or co we define h(z) = g(±^/z) and note that h is well defined (regardless of the 
branch of the square root taken) since g is even. Since h is analytic on C \ {0} and can be extended 
in the obvious way to be continuous on C, h is rational and satisfies h(z 2 ) = g(z). □ 

Lemma 2.5. Let f be a rational map such that f(U) = U . Then there exists a rational map f such 
that 4> o f = f o (jj if and only if f is odd (and therefore of the form in Lemma \2.1\) . 
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Proof. Let / be odd. Since f(z) = —f(—z) we see that [f(z)} 2 is an even rational function and 
therefore by Lemma \2A\ [f(z)] 2 = h(z 2 ) for some rational map h. Define f(z) — ^ \+l (hence 
/ is a rational map as it is a composition of rational maps). Let w = 4>{z) = fsfrj and note that 
z 2 = i+H. Hence (/ o 4>)(z) = f(w) = = f^ff^ = (0 o f)(z). 

Suppose there exists a rational map / such that <f> o / = / o <j). Then / o <f> is even since cj> is even. 
The semi-conjugacy implies (j> o / is also even, which by Lemmas 12.31 and 12.21 gives that / is either 
even or odd. If / were even, then /(C \ U) = f(U) = U and the preimage of the lower half plane 
would be empty. This contradicts the fact that the image of C under a rational map is always C. 
Hence we conclude that / must be odd. □ 

Lemma 2.6. If f is a rational map such that Q = / _1 (ri), then there exists an odd rational map f 
such that U = f~ l {U) and (f> o / = / o (f>. 

Proof. Let h denote the branch of the inverse of <f> which maps f2 onto U. Then / = h o / o <f> 
maps U onto U properly and is therefore a rational map (Blaschke product of the upper half plane). 
Clearly, <pof~fo<j)onU and so by the identity Theorem this semi-conjugacy holds on all of C. 
By Lemma 12.51 f is odd. □ 

Remark 2.1. Lemmas 12.51 and 12.61 classify those rational functions that can be semi-conjugated by 

4>. 

Lemma 2.7. For rational semigroups G = (gj : j € T) and H = (hj : j £ I) where there exists 
a rational function k satisfying the semi-conjugacy relation k o hj = gj o k for each j £ X, we have 
J(G) = k(J(H)) and N(G) = k(N(H)). 

Proof. We first note that the semi-conjugacy relation on the generators translates to a semi-conjugacy 
relation between corresponding elements of the semigroups. More precisely, if h — hj t o • • • o hj n € H , 
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then for g = g^ x o • • • o gj n we have k o h = g o k since k o hj 1 o • • • o ft,j n = o k o ft,j 2 o • • • o = 
°9h°koh h o--.ohj n = = g n o g h o ■ ■ ■ o g jn o k. 

Let zq be a point in N(H) and let A be a small open set in N(H) containing zq such that h(A) 
has spherical diameter less than e for all h £ H. Denoting the Lipschitz constant (with respect 
to the spherical metric) of k by C (see [2J, p. 32), we see that for any g = g^ o • • • o gj n e G the 
diameter of g(k(A)) — k(hj 1 o • • • o hj n (A)) = /c(/i(A)) is less than Ce. Hence k(zo) S N(G) and so 
we conclude that k(N(H)) C iV(G). 

Let £o be a repelling fixed point for some h — hj t o • ■ ■ o hj n G H, but which is not a critical point 
of k. Then for g = g^ o ■ ■ ■ o we have g o k — k o h and hence g has a fixed point at fc(zo) with 
the same multiplier as that of h at zq (using the chain rule and the fact that g = k o h o k^ 1 for the 
branch of k^ 1 which maps fc(zo) to zq). Hence we have shown that the repelling fixed points of the 
maps in H, which are not any of the finite number of critical points of k, map under k to repelling 
fixed points of maps in G. Since the Julia set of a rational semigroup is a perfect set equal to the 
the closure of the set of repelling fixed points of the elements of the semigroup (see [3], Theorem 3.1 
and Corollary 3.1), it then follows that k(J(H)) C J(G). 

Since J(H) = C \ N(H) and J(G) = C \ N(G) the lemma now follows from the fact that 
k(C) = C. □ 

One might expect that a result similar to Lemma 12.71 would hold for completely invariant Julia 
sets, however, we require an additional hypothesis as noted in the following lemmas. 

Lemma 2.8. Suppose rational functions g,h,k satisfy the semi-conjugacy relation koh = g ok. 
If S is completely invariant under g, then k~ 1 (S) is completely invariant under h. Also, if S is 
completely invariant under h and k~ 1 (k(S)) — S, then k(S) is completely invariant under g. 

The proof of Lemma 12.81 follows readily from the semi-conjugacy and will therefore be omitted. 
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Lemma 2.9. For rational semigroups G = (gj : j £ I) and H = (hj : j £ I) where there exists 
a rational function k satisfying the semi-conjugacy relation k o hj = gj o k for each j £ X, we have 
k(E(H)) C E(G) (and thus W(G) C k(W(H))). If we also have that fc- 1 (&(£(#)) = E(H), then 
k(E(H)) = E(G) and W(G) = k{W{H)). 

Remark 2.2. The hypothesis k^ 1 (k(E(H))) — E(H) stated above would automatically follow from 
the other assumptions if, in addition, k is a (branched) Galois covering. We, however, do not require 
that form of the statement because one can easily check that this hypothesis holds in the situations 
we consider below. 

Proof. Let h — hj 1 o ■ • ■ o hj n £ H and consider the corresponding g = gj 1 o ■ • ■ ogj n £ G. Since E(G) 
is completely invariant under g and k o h = g o k, Lemma [2T81 shows that the closed set k^ 1 (E(G)) 
is completely invariant under h. Since h £ H was arbitrary, we conclude that E(H) C k^ 1 (E(G)). 
Thus k(E(H)) C E(G). 

Similarly one can use Lemma [2T8l to show that k^ 1 (k(E(H)) = E(H) implies E(G) C k{E(H)) 
and so E(G) = k(E(H)). When fc- 1 (fc(^(i7)) = E(H), k maps E{H) in a deg(fc)-to-one fashion 
onto k(E(H)) = E(G). Since k is a rational map of global degree deg(fc), it must then map 
W(H) = C \ E(H) onto W(G) = C \ E(G) (also in a deg(fc)-to-one fashion). □ 

Example 2.1 (Counterexamples to Conjectures 11.11 and 11.21 ). Let / be an odd rational map such 
that f(U) = U . Then by Lemma [2~5l there exists a rational function / satisfying the semi-conjugacy 
relation <f> o / = / o cf>. Similarly we let g be an odd rational map with g(U) — U and so there exists 
a rational map g with <f> o g = g o <j). By choosing / and g such that Jf ^ R and J g = R, we have 
that J j ^ I and Jg = I by Lemma 12.71 Since R is completely invariant under both / and g we have 
E{G) C I where G = (f,g). Since E{G) D J 5 = R, we conclude that E(G) = R. For G = 
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we see that since <j)~ 1 {(p{E{G))) = (<£(!)) = R = E(G), we must have E(G) = 0(1) = /. Since 
Jf 7^ Jg, G is a counterexample to Coniectures ll.il and 11.21 

Specifically we may select f(z) = 2z — | and g(z) = z 2 ~ 1 . Hence Jf is a Cantor subset of / 
(see [2], p. 21). Since g is the conjugate of z >— > z 2 under z i— > ijlrf we see that J s = R. In this case 
one can calculate (via the proof of Lemma \2~5\i that f(z) — frfj^rf^ and g(z) = 2z 2 — 1. 

In the next example, we construct a semigroup G that provides a counterexample to Conjec- 
tures [Ll] and [L2] with the additional property that J(G) C E(G). 

Example 2.2. Consider /(z) = 2z — l/z as in Example 12. 11 Let (p(z) = 2z, and set g(z) — (tp o / o 
99 _1 )(z) = 2z — A/z. Note that J g = f{Jf) = 2Jf and that K is completely invariant under g. Hence 
for G = (/, g), we have E(G) C I. 

Suppose that E(G) ^ K. Since K is completely invariant under both / and it follows from 
Lemma 3.2.5 in [5j that if E(G) contains a non-degenerate interval in the real line, then E(G) = K. 
Hence we may select an open interval L — (x, y) in R \ E(G) with both x, y large. Since the length 
of the intervals f n (L) tends to +oo, we may assume that y — x is large. By expanding the interval 
we may also assume that x,y S E(G) (note that we used here that oo is a non-isolated point in 
E(G) which follows since 2 e J g C .5(G) and /"(2) -> oo). 

Since a; is large, we can use the fact that f(x) is slightly greater than g{x) to see that 3 _1 ({/(2:)}) 
contains a point slightly larger than x (and hence less than y). But by the complete invariance of 
the set E(G) under / and g, we get 5~ 1 ({/(a;)}) C E(G). This is a contradiction since the interval 
(x, y) does not meet E(G). We conclude that E(G) = E. 

Since oo is an attracting fixed point under both / and g, we see that small neighborhoods of oo 
map inside themselves under each map in G. Hence oo £ N(G) and so J(G) ^ R. 

As in Example l2.1l we may semi-conjugate the odd rational maps / and g by <f> to get maps f(z) = 
and = 3^+40^27 - Hence for G = (f,g) we have J(G) = cj>{J{G)) C 0(R) = / and 
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E(G) = (j>{E{G)) = 0(1) = I. Since Jj ^ J g (otherwise one would have E(G) = J(G) = Jf = J g ), 
we see that G is a counterexample to Coni ect ures 11.11 and 11.21 

3. Counterexamples to Conjecture 11.41 
In [3J, p. 366 Hinkkanen and Martin give the following definition. 

Definition 3.1. A rational semigroup G is nearly abelian if there is a compact family of Mobius 
transformations $ = {</>} with the following properties: 

(i) 4>(N(G)) = N(G) for all </> e $, and 

(ii) for all f,g & G there is a S $ such that f°g = 4>°g°f- 

Theorem 3.1 ([3], Theorem 4.1). Lei G be a nearly abelian semigroup. Then for each g € G we 
have J g = J(G). 

A natural question is to what extent does the converse to Theorem 13.11 hold. Using a result 
of A. Beardon (see pQ, Theorem 1) Hinkkanen and Martin have proved the following result for 
polynomial semigroups. 

Theorem 3.2 ([3], Corollary 4.1). Let J- be a family of polynomials of degree at least 2, and suppose 
that there is a set J such that J g = J for all g G T ' . Then G = (J-) is a nearly abelian semigroup. 

Note that under the hypotheses of Theorem l3.2l we have Jh — J(G) for each generator h € T . So 
we see that Conjecture [L4] is suggesting that if Jh = J{G) for just one h £ G, then G is still nearly 
abelian. However, this is not the case as we see by the following counterexample. 

Example 3.1 (Counterexample to Conjecture [OJ. Let f(z) — z 2 — 2, g(z) — 4z 2 — 2 and G — (/, g). 
It is well known that / is a conjugate of 2z 2 — 1 by z i— > 2z and so Jf = [—2,2] (see [2], p. 9). 
It can easily be seen that g maps [—1, 1] onto [—2,2] in a two-to-one fashion. Since g~ 1 ([— 1, 1]) C 
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.g _1 ([-2, 2]) = [-1, 1] it follows that J g C [-1, 1]. In particular J g C J f . We also note that C\ [-2, 2] 
is forward invariant under both / and g and as such must lie in N(G) by Montel's Theorem. It 
follows that J(G) = [-2, 2] = Jf, yet J f ^ J g . 

We remark that any map g that maps a proper sub-interval of [—2, 2] onto [—2,2] in a deg(g)- 
to-one fashion would suffice in the above example and such functions can easily be obtained by 
constructing real polynomials with appropriate graphs. Also, / may be replaced by any Tchebycheff 
polynomial (see section 1.4 of [2]), normalized so that Jf = [—2,2]. 

Acknowledgements: The authors would like to thank Aimo Hinkkanen for his advice given during 
the preparation of this manuscript. 
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